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DETAILED ACTION 

1 . The response received on 13 January 2005 has been placed in the file and was considered by the examiner. 
An action on the merits follows. 

Response to Arguments 

2. The arguments filed on 13 January 2005 have been fully considered. A response to these arguments is 
provided below. 

Specification Objections 

Summary of Argument : 

Applicant has amended the specification in view of the examiner's objections. 

Examiner 's Response : 

The specification objections have been corrected by amendment and are therefore withdrawn. 

Claim Objections 

Summary of Argument : 

Applicant has amended the claims in view of the examiner's objections. 
Examiner 's Response : 

The claim objections have been corrected by amendment and are thus withdrawn. 

35 USC 112, Second Paragraph Rejections 

Summary of Argument : 

Claims 1-7, 11-12, and 20 stand rejected under 35 USC 112(2) as being indefinite. Applicant has amended all of 
the claims in question, and argues that the rejections should be withdrawn (remarks, pg, 12). 

Examiner 's Response : 

Applicant's amendment has overcome the previous rejection for all claims except claim 20. The rejection to 
claim 20 will be repeated below. 

Double Patenting 

Summary of Argument : 

Applicant has cancelled claims 13-15 as being substantial duplicates of claims 8-10. 
Examiner 's Response : 

The previous objection is withdrawn. 



Application/Control Number: 09/895,063 
Art Unit: 2621 



Page 3 



Prior Art Rejections 

A rguments and Responses : 

1 . Regarding claim 12, Applicant has amended this claim to emphasize that construction under 35 USC § 1 12(6) 
("means plus function") is intended. Applicant alleges that with this more narrow interpretation, the cited art does 
not render obvious the arrangement defined by claim 12 (remarks, pg. 13). 

response : 

Applicant's intention to invoke § 1 12(6) is noted. The examination of claim 12 will proceed accordingly. 

2. Regarding independent claim 17, Applicant has amended this claim to include the previous claim 18 
"watermark" limitation. Applicant alleges the following: 

(a) the combination of Honjoh and Tian is based on hindsight reasoning; 

(b) said combination leads to a conclusion not required by the claim; and 

(c) that Tian alone does not teach the use of a watermark signal to align two or more images to each other. 
Specifically, applicant argues that "[Tian's] watermark is used to determine rotation, scale and rotation to which a 
single image has been subjected after encoding, so that these geometric operations can be taken into account prior to 
decoding of that single image." 

response : 

(a) In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper 
hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a 
reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was 
within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge 
gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 
170 USPQ 209 (CCPA1971). 

(b) This argument is irrelevant. 

(c) This argument is also irrelevant. The rejection was based on a combination of references, not the teachings 
of a single reference. 

3. Regarding claim 20, applicant states that "No art-based rejection appears to have been made of claim 20." 
response : 

This is incorrect. An art-rejection was made to claim 20 on page 13 (paragraph 17) of the previous office action. 
It does appear that the heading of this paragraph was mislabeled ("Claim 17 is rejected"). But the body of the 
paragraph explicitiy refers to claim 20 and expressly addressed its limitations. 
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Claim Rejections - 35 USC § 112 

3. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

4. Claims 6, 7, 20 are rejected under 35 U.S.C. 112, second paragraph, as being indefinite for failing to 
particularly point out and distinctly claim the subject matter which applicant regards as the invention. 

Claim 20 is indefinite for failing to identify the environment of the method (the preamble of the claim) versus the 
steps of the method. This claim does not recite a transition such as "comprising" or "consisting of in order to 
identify the end of the preamble and the start of the steps. It appears that this claim is simply a preamble without a 
body. 

Claim 6 is indefinite because it refers to "said determined low resolution images." This lacks antecedent basis in 
the claim. 

Claim 7 is indefinite for depending on an indefinite claim. 

Allowable Subject Matter 

5. Claims 1-5, 11, 16, 19, 28, 29 are allowed. 



Claim Rejections - 35 USC §102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the 
rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

7. Claims 6-9, 21-27, and 30-32 are rejected under 35 U.S.C. 102(e) as being anticipated by Messing et al 
(USPN 6,466,618). 

Regarding claim 6, Messing discloses generating a high resolution image from a pluraltiy of low resolution 
images (abstract, first line) whose pixel values are in Bayer squares (col. 6 lines 17-20). 

Messing further discloses identifying a first group of said low resolution images that align with a first pixel hole 
position of a Bayer square (col. 6 lines 22-25 in conjunction with Fig. 7: The reference describes identifying a blue 
color field (i.e. a first group) that aligns with a blue sub lattice (i.e. first pixel hole position of a bayer square). 
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Messing further discloses identifying a second, different, group of said low resolution images that align with a 
second, different, pixel hole position of the Bayer square (col. 6 lines 22-25 in conjunction with Fig. 7: The 
reference describes identifying a red color field (i.e a second group) that aligns with a red sub lattice (i.e. second 
pixel hole positions of a bayer square).). 

Messing further discloses combining said low resolution images to fill in the holes in a Bayer square (col. 7 lines 
39-44 and elsewhere in the specification in conjunction with Figs. 9 and 10). 

The examiner would like to respectfully remind applicant that any of a laundry list of references are easily 
applied to this claim. This claim essentially recites the basics steps of a Bayer filter — which have been known since 
1976. 

Regarding claim 7, Messing discloses that the pixel values in multiple images which are aligned with each pixel 
position are averaged and the averaged images are combined to fill the holes in a bayer square (Messing col. 5 lines 
57-67: The reference describes a step called "color filter array interpolation" which is used to combine the imags. 
Interpolation — as is well known — is an averaging operation. The multiple images in Messing are therefore average 
as is required by the claim.). 

Regarding claim 8, Messing discloses capturing a plurality of low resolution electronic images of a subject, the 
subject defining a hidden reference signal (col. 3 lines 21-25: The coordinate system of the captured low resolution 
image is a hidden reference signal.). 

Messing discloses usng said reference signal to determine alignment of a plurality of said low resolution images 
(col. 4 lines 24-36). 

Mess discloses combining data from some but not all of said low resoluton images into a high resolution image 
(col. 9 lines 10-24). 

Regarding claim 9, Messing discloses that the low resolution imags are aligned in accordance with the holes in a 
Bayer square (see Fig. 7 in conjunction with col. 6 lines 18-42). 

Regarding claim 21, Messing discloses capturing first and second frames of similar image data using a common 
2D image sensor that is subject to slight movement between frames, said sensor comprising a geometrical pattern of 
sensor elements, each element sensing light of one of at least first or second colors, wherein certain locations in said 
2D sensor do not sense light of the first color, resulting in unknown first color information at various locations in the 
frames of image data captured by said sensor (Messing col. 6: The reference describes a bayer pattern. Thus we 
know that images captured from a red sensor e.g. will have certain locations that are unknown. This is how the 
Bayer pattern works. Red is sampled less often, so certain locations won't have sensed the color. Messing further 
discloses that this sensor is subject to slight movement between frames (col. 9 lines 2-6). Obviously since the 
reference discloses movement between frames, the reference also teaches the capturing of a first and second frame.). 

Messing further discloses determining position information relating position of the first image frame to the 
second image frame (col. 9 lines 9-17: The reference describes motion data that relates the positions of two frames). 

Messing further discloses generating a composite image frame in which said unknown first color information at 
one of said various locations in the first frame of image data is mitigated by reference to first color information from 
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the second image frame that is determined to correspond to said one location (col. 8 lines 56-65; col. 9 lines 6-8; col. 
6 lines 59-63: The reference describes that the shifting of the images with respect to one another allows for the 
insertion of previously unknown data.). 

Regarding claim 22, Messing discloses said unknown color information in the first frame is thereby estimated 
without resort to interpolation (col. 9 lines 6-9). 

Regarding claim 23, Messng discloses determining comprises detecting hidden information in said first and 
second image frames (Messing discloses that the determination of position comprises detection of the coordinate 
system (i.e. hidden information) in both frames.. Indeed, all the images detect this hidden information. It would be 
impossible to detectio motion without some frame of reference.). 

Regarding claim 24, Messing discloses correlating with a pattern in said first and second image frames to 
detemine position (col. 6 lines 51-54: The reference describes a comparison of the luminance of the two frames. 
This luminance qualifies as the claimed pattern. 

Regarding claim 25, Messing discloses that the determining makes reference to fiducial markings (see Figs. 3, 5, 
and 10: Fiducials are merely reference points, so the markings on the image or the reference coordinates from 
Messing qualify as the claimed fiducial). 

Regarding claims 26 and 27, Messing discloses both a regular pattern and a Bayer pattern (col. 6 lines 8-42). 

Regarding claim 30, Messing discloses identifying a marker signal in each of said low resolution images to relate 
locations of said images (col. 3 lines 22-25). 

Messing further discloses identifying a first group of said low resolution images having image information of a 
first primary color at a location most closely aligned with a first position in said high resolution image (col. 8 lines 
40-42). 

Messing further discloses identifying a second group of said low resolution images having image information of 
a second, different, primary color at a location most closely aligned with said first position in said high resolution 
image (col. 8 lines 61-65). 

Messing further discloses processing the image information of the first primary color from said first group of 
images to generate data of said first primary color for the first position in said high resolution image (col. 8 lines 50- 
55). 

Messing further discloses processing the image information of the second primary color from said first group of 
images to generate data of said second primary color for the first position in said high resolution image (col. 9 lines 
5-8). 

Regarding claim 3 1, Messing discloses that none of the low resolution images in the first group of images is 
included in the second group of images. Indeed they are two separate colors. 

Regarding claim 32, messing discloses that at least one of said low resolution images is included in neither said 
first nor second groups. Indeed, there are three primary colors. 



8. 



Claim 12 is rejected under 35 U.S.C. 102(e) as being anticipated by Howell (USPN 6,570,613). 
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Howell discloses a system for generating a high resolution image from a plurality of relatively low resoluton 
images whose pixel values are in a Bayer square configuration (col. 9 lines 48-60: The reference describes that the 
CFA is a bayer pattern CFA). 

Howell further discloses means for determining which of said images align with each pixel position of a Bayer 
square to within a specified tolerance (col. 9 lines 16-26: The reference describes a specified tolerance of a one 
pixel shift. This is equivalent ot the one pixel shift discloed in applicant's specification). 
Howell further discloses means for combining multiple aligned low resolution images to fill in holes in a Bayer 
square (see e.g. Fig. 1 1 : Again, the reference is sufficient to meet the claim limitation even with 112 sixth paragraph 
invoked.). 

Claim Rejections - 35 USC § 103 

9. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Honjoh in view of Tian et al. 
Referring to claim 17, Honjoh discloses 

i. Capturing a series of low resolution images, each of which contain a reference signal is illustrated by 
Honjoh in figure 3 by the sequential still images. 

ii. Reading the reference signal from each of the low resolution images is explained by Honjoh in 
column 7, lines 12-23 wherein the extracted contours are used to create a characteristic pattern 
(corresponding to the reference signal) in order to determine the relative displacement amounts for each 
frame (corresponding to the alignment of the pixels in the image). Honjoh illustrates in figure 3 that the 
characteristic pattern is determined from each of the sequential still images, corresponding to the low 
resolution images as explained by Honjoh in column 1, line 62 to column 2, line 12. 

iii. Aligning the low resolution images in accordance the location of the reference signal is explained by 
Honjoh in column 8, lines 4-14 wherein the images that are substantially overlapping (within A(n t )) are 
used to create a high resolution image. 

iv. Combining the aligned low resolution images into a high resolution image is illustrated by Honjoh in 
figure 3 by "combine frames". 

Further referring to claim 17, the reference signal being a watermark signal is not explicitly explained by 
Honjoh. The system of Honjoh uses the characteristic pattern of the contours as the reference signal. However, 
Tian et al explain that a watermark orientation signal is used to align image data in column 11, lines 48-60. Tian et 
al explain that watermarks are imperceptible, but may be detected by an appropriate decoder in column 1, lines 21- 
33. Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was made to 
use a watermark as a reference signal in the system of Honjoh and Glotzbach et al, as suggested by Tian et al, 
because the reference signal would provide a second imperceptible signal to align the images more accurately. 
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10. Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bergen (U.S. Patent No. 6,208,765) 
in view of Tian et al. 

Referring to claim 20, aligning multiple low resolution images to form a high resolution image wherein both a 
hidden reference signal embedded in the low resolution images and visible image content are used to align said 
images is not explicitly explained by Bergen. Bergen does explain aligning multiple low resolution images (figure 
2) using the visible image information in column 3, lines 1 1-25. However, Bergen does not explain using a hidden 
reference signal. Tian et al explain that a watermark orientation signal is used to align image data in column 1 1, 
lines 48-60. Tian et al explain that watermarks are imperceptible, but may be detected by an appropriate decoder in 
column 1, lines 21-33. Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to use a watermark as a reference signal in the system of Bergen, as suggested by Tian et al, 
because the hidden reference signal would provide a second imperceptible signal to align the images more 
accurately. 

1 1 . Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Messing (USPN 6,466,618) in view 
of Wolf (USPN 5,767,987) 

Referring to claim 10, a plurality of low resolution images being captured and only those low resolution images 
which align to within a specified tolerance with the holes in a Bayer square being used to form said composite image 
is not explicitly explained by Messing. Messing discloses an optimum tolerance for low resolution images should 
fall within (Messing col. 9 lines 1-10), but fails to expressly disclose that those that do not fall within this tolerance 
are discarded. However, Wolf discloses that there is a certain range that displacement must fall within (Wolf col. 7 
lines 20-25). It would have been obvious to one reasonably skilled in the art at the time of the invention to modify 
Messing 's Bayer square aligning method by introducing an offset tolerance as taught by Wolf. Such a modification 
would have allowed for a more efficient system that discarded images to minimize error (Wolf col. 6 line 60 - col. 7 
line 25). 



Conclusion 

12. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. 
Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension 
of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the 
mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final 
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action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, 
then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee 
pursuant to 37 CFR 1. 136(a) will be calculated from the mailing date of the advisory action. In no event, however, 
will the statutory period for reply expire later than SIX MONTHS from the date of this final action. 

13. Any inquiry concerning this communication or earlier communications from the examiner should be 
directed to Patrick L Edwards whose telephone number is (571) 272-7390. The examiner can normally be reached 
on 8:30am -5:00pm M-F. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Joe Mancuso 
can be reached on (571) 272-7695. The fax phone number for the organization where this application or proceeding 
is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent Application Information 
Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR 
or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more 
information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the 
Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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